ggbetweenstats( data =samdat_tbl(ps), x =diagnosis, y =N_genera, type ="parametric", p.adjust.method ="fdr", var.equal =TRUE, bf.message =FALSE, results.subtitle =TRUE)
4.2 Diversity
Remember: a true measure of ecosystem diversity (e.g. Shannon index) will consider the richness and evenness of the ecosystem.
A rich ecosystem dominated by only one or two of its taxa is still a less diverse ecosystem than one with an even distribution of the same number of taxa.
We already computed Shannon diversity of genera and the effective Shannon in part 1B.
Here we will use the effective Shannon diversity \(e^H\) because of its more intuitive interpretation.
ggbetweenstats( data =samdat_tbl(ps), x =diagnosis, y =Effective_Shannon_Genus, type ="parametric", p.adjust.method ="fdr", var.equal =TRUE, bf.message =FALSE, results.subtitle =TRUE)
5 Dissimilarity & Ordination
Our research questions for this section are:
What gut bacterial microbiota compositions are present or common in this cohort?
Does the average overall composition of the gut microbiota differ by patient diagnosis group?
5.1 Aggregate ➔ Calculate ➔ Ordinate
In order to create a PCoA ordination - we need to first make two choices:
At which taxonomic rank will we aggregate the counts? (for 16S data, this is usually Genus)
Which dissimilarity measure to use when calculating the distance matrix?
5.2 Common dissimilarity measures
You heard about several commonly-used dissimilarity measures in the lecture. In the sections below, we will calculate a distance matrix with each one, and use each to plot a PCoA and test for diagnosis group differences with PERMANOVA.
Permutation test for adonis under reduced model
Marginal effects of terms
Permutation: free
Number of permutations: 999
vegan::adonis2(formula = formula, data = metadata, permutations = n_perms, by = by, parallel = parall)
Df SumOfSqs R2 F Pr(>F)
diagnosis 2 1.5068 0.05441 2.5032 0.001 ***
Residual 87 26.1846 0.94559
Total 89 27.6914 1.00000
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
The phylogenetic distance family - has unweighted, weighted, and generalised versions.
You must use ASV-level data (i.e. no taxonomic aggregation) and have a phylogenetic tree available.
We will not practice with UniFrac distances today, because they can be quite slow to calculate.
The Aitchison distance is a CoDA distance method - named after John Aitchison, a pioneer in the field of Compositional Data analysis. related to CLR + PCA.
Permutation test for adonis under reduced model
Marginal effects of terms
Permutation: free
Number of permutations: 999
vegan::adonis2(formula = formula, data = metadata, permutations = n_perms, by = by, parallel = parall)
Df SumOfSqs R2 F Pr(>F)
diagnosis 2 2552 0.10114 4.8946 0.001 ***
Residual 87 22681 0.89886
Total 89 25233 1.00000
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
You can see from the PERMANOVA model outputs that the p value is below 0.05. So there is good statistical evidence that the bacterial gut microbiota composition of C-section delivered infants has a different composition than vaginally delivered infants at 4 days of age.
Dissimilarity or distance?
These terms are often used interchangeably.
Strictly, all distances are dissimilarities, but not all dissimilarities are distances.
A true “distance metric” \(d\), must satisfy 3 properties:
Identity of indiscernibles: For any samples \(a\) and \(b\), \(d(a, b) = 0\) if and only if \(a = b\)
Symmetry: For any samples \(a\) and \(b\), \(d(a, b) = d(b, a)\)
Triangleinequality: For any samples \(a\), \(b\), and \(c\), \(d(a, c) ≤ d(a, b) + d(b, c)\)
3 means: “the direct path between two points must be at least as short as any detour”
3 is not true for e.g. Bray-Curtis… but in practice this is very rarely problematic
psExtra objects
microViz often creates objects of class psExtra which store info about the aggregation and transformations you perform.
psExtra can also store a distance matrix (and an ordination or PERMANOVA results)
You can extract the distance matrix with dist_get()
e.g. use the data of only the UC patients’ gut microbiota!
Colour points by the dominant genus?
# Copy these lines to your Consoleps%>%tax_filter(min_prevalence =2, verbose =FALSE)%>%ps_filter(diagnosis=="UC")%>%# calculate dominant Genus for each sample (optional)ps_calc_dominant(rank ="Genus", none ="Mixed", other ="Other")%>%ord_explore()
Beware: some important notes on interactive analysis
There are many distances available
Feel free to try out distances we haven’t talked about, BUT:
You should not use a distance that you don’t understand in your work, even if the plot looks nice! 😉
A few of the distances might not work correctly…
They are mostly implemented in the package vegan and I haven’t tested them all
Errors may appear in the RStudio Console
You can report to me any distances that don’t work (if you’re feeling helpful! 😇)
There are several other ordination methodsavailable
Try out PCA, principal components analysis, which does NOT use distances
We will not discuss “constrained” or “conditioned” ordinations today:
2024-06-29 15:38:56.36597 - Starting PERMANOVA with 99 perms with 1 processes
2024-06-29 15:38:56.418699 - Finished PERMANOVA
Permutation test for adonis under reduced model
Marginal effects of terms
Permutation: free
Number of permutations: 99
vegan::adonis2(formula = formula, data = metadata, permutations = n_perms, by = by, parallel = parall)
Df SumOfSqs R2 F Pr(>F)
diagnosis 2 1.4591 0.05269 2.4103 0.01 **
gender 1 0.1557 0.00562 0.5145 0.98
age_years 1 0.2899 0.01047 0.9577 0.51
Residual 85 25.7279 0.92909
Total 89 27.6914 1.00000
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Use more permutations for a more precise and reliable p.value in your real work (it is slower).
Always set a seed number for reproducibility of this random permutation method!
Reporting PCoA and PERMANOVA methods
Your methodological choices matter, you should report what you did:
any relevant rare taxon filtering
the taxonomic rank of aggregation
the dissimilarity measure used to compute the pairwise distances
any covariates included in the statistical model
It’s a good idea to decide on one or two distance measures a priori, and report both (at least in supplementary material). The choice of distance measure can affect results and conclusions!
Interestingly, samples on the right of the plot (which tend to be UC patients) seem to have relatively more Escherichia/Shigella, and less Blautia, Faecalibacterium and Roseburia.
Wait, how to interpret these taxa loadings?
In general:
The relative length and direction of an arrow indicates how much that taxon contributes to the variation on each visible PC axis, e.g. Variation in Faecalibacterium abundance contributes quite a lot to variation along the PC1 axis.
The direction allows you to infer that samples positioned towards the left of the plot will tend to have higher relative abundance of Faecalibacterium than samples on the right of the plot.
Bacteroides variation contributes to both PC1 and PC2, as indicate by its high (negative) values on both axes.
Cautiously
There are caveats and nuance to the interpretation of these plots, which are called PCA bi-plots
The sequencing data gives us relative abundances, not absolute abundances.
The total number of reads sequenced per sample is an arbitrary total.
This leads to two main types of problem:
Interpretation caveats: see differential abundance section later
Statistical issues: taxon abundances are not independent, and may appear negatively correlated
These issues are worse with simpler ecosystems
Example: If one taxon blooms, the relative abundance of all other taxa will appear to decrease, even if they did not.
There is the same problem in theory with RNAseq data, but I suspect it is less bothersome because there are many more competing “species” of RNA transcript than there are bacterial species in even a very complex microbiome.The centered-log-ratio transformation (along with some other similar ratio transformations) are claimed to help with the statistical issues by transforming the abundances from the simplex to the real space.
TL;DR - the CLR transformation is useful for compositional microbiome data.
Practically, the CLR transformation involves finding the geometric mean of each sample
Then dividing abundance of each taxon in that sample by this geometric mean
Finally, you take the natural log of these ratios
ps%>%tax_sort(by =sum, at ="Genus", trans ="compositional", tree_warn =FALSE)%>%tax_agg(rank ="Genus")%>%tax_transform(trans ="clr", zero_replace ="halfmin", chain =TRUE)%>%comp_heatmap( samples =1:20, taxa =1:20, grid_lwd =2, name ="CLR", colors =heat_palette(sym =TRUE), tax_seriation ="Identity")
:::
Fancy circular bar charts?
We can make another kind of bar plot, using the PCA information to order our samples in a circular layout.
From the PCA loadings and barplots below, we have some strong suspicions about which taxa have a higher relative abundance in vaginally delivered infants than in c-section delivered infants, and vice versa, but we can also statistically test this. This is often called “differential abundance” (DA) testing, in the style of “differential expression” (DE) testing from the transcriptomics field.
This method simple method is borrowed from MaAsLin2
Note: they call the compositional transformation “Total Sum Scaling (TSS)”)
This is quite a straightforward method, so we will stick with this for today
But, many statistical methods have been developed for differential abundance analyses
Microbiome abundance data are quite awkward, statistically speaking, due to their sparseness and compositionality. Each successive method claims to handle some aspect of this awkwardness “better” than previous methods.
The aim is to have a method with adequate power to detect true associations, whilst controlling the type 1 error rate, the “false positive” reporting of associations that are not “truly” present.
Filter out the noise & interpret results with caution! use multiple testing corrections
Remember it’s all relative (abundance)
Try 2 or 3 methods and/or use same method as a previous study if replicating
Avoid Lefse and edgeR?
Beware: Not all methods allow covariate adjustment & few allow random effects (for time-series)
10.4 Now model all the taxa!?
We’re not normally interested in just one taxon!
It’s also hard to decide which taxonomic rank we are most interested in modelling!
Lower ranks like species or ASVs give better resolution but also more sparsity and classification uncertainty…
Higher ranks e.g. classes, could also be more powerful if you think most taxa within that class will follow a similar pattern.
So now we will fit a similar model for almost every taxon* at every rank from phylum to genus
*We’ll filter out species with a prevalence of less than 10%
# The code for `taxatree_models` is quite similar to tax_model.# However, you might need to run `tax_prepend_ranks` to ensure that each taxon at each rank is always unique.psModels<-ps%>%tax_prepend_ranks()%>%tax_transform("compositional", rank ="Genus", keep_counts =TRUE)%>%tax_filter(min_prevalence =0.1, undetected =0, use_counts =TRUE)%>%taxatree_models( type =lm, trans ="log2", trans_args =list(zero_replace ="halfmin"), ranks =c("Phylum", "Class", "Order", "Family", "Genus"), variables =c("IBD", "Female", "Age_Z"))
psModels
psExtra object - a phyloseq object with extra slots:
phyloseq-class experiment-level object
otu_table() OTU Table: [ 65 taxa and 90 samples ]
sample_data() Sample Data: [ 90 samples by 23 sample variables ]
tax_table() Taxonomy Table: [ 65 taxa by 6 taxonomic ranks ]
otu_get(counts = TRUE) [ 65 taxa and 90 samples ]
psExtra info:
tax_agg = "Genus" tax_trans = "compositional"
taxatree_models list:
Ranks: Phylum/Class/Order/Family/Genus
Why filter the taxa? It’s less likely that we are interested in rare taxa, and models of rare taxon abundances are more likely to be unreliable. Reducing the the number of taxa modelled also makes the process faster and makes visualizing the results easier!
Notes on filtering rare taxa
We probably want to filter out rare taxa, before performing some kinds of analysis.
Why remove rare taxa?
Rare taxa might sometimes be:
Sequencing errors
Statistically problematic
Biologically irrelevant
How to remove rare taxa?
What is rare? Two main concepts.
Low prevalence - taxon only detected in a small number of samples in your dataset.
Low abundance - relatively few reads assigned to that taxon (on average or in total)
Considering the impact of issues 1, 2, and 3, let’s say we are not interested in Species that occur in fewer than 2% of samples, and they have to have at least 10,000 reads in total across all samples.
ps%>%tax_filter(min_prevalence =2, min_total_abundance =100)%>%ntaxa()# after filtering
[1] 299
Wow so that would remove most of our unique taxa!
What is going on? Let’s make some plots!
# make table of summary statistics for the unique taxa in shao19psTaxaStats<-tibble( taxon =taxa_names(ps), prevalence =microbiome::prevalence(ps), total_abundance =taxa_sums(ps))
Warning: The `trans` argument of `sec_axis()` is deprecated as of ggplot2 3.5.0.
ℹ Please use the `transform` argument instead.
Some ggplot2 code
p
So most taxa have a low prevalence, and handful have way more reads than most.
Let’s label those points to check which taxa are the big time players.
p+ggrepel::geom_text_repel( data =function(df)filter(df, total_abundance>1500|prevalence>0.6), mapping =aes(label =taxon), size =2.5, min.segment.length =0, force =20, nudge_y =0.05)
Those taxa make sense for this dataset of gut microbiota samples.
Now let’s zoom in on the less abundant taxa by log-transforming the axes. We’ll also add lines indicating the thresholds of 2% prevalence and 1000 reads abundance.
Warning: There was 1 warning in `mutate()`.
ℹ In argument: `Phylum = forcats::fct_explicit_na(Phylum, na_level = "Other")`.
Caused by warning:
! `fct_explicit_na()` was deprecated in forcats 1.0.0.
ℹ Please use `fct_na_value_to_level()` instead.
Next we will get a data.frame containing the regression coefficient estimates, test statistics and corresponding p values from all these regression models.
We have performed a lot of statistical tests here!
It is likely that we could find some significant p-values by chance alone.
We should correct for multiple testing / control the false discovery rate or family-wise error rate.
Instead of applying these adjustment methods across all taxa models at all ranks, the default behaviour is to control the family-wise error rate per taxonomic rank.
But how do we know which taxa are which nodes? We can create a labelled grey tree with taxatree_plotkey(). This labels only some of the taxa based on certain conditions that we specify.
set.seed(123)# label positionkey<-psStats%>%taxatree_plotkey( taxon_renamer =function(x)stringr::str_remove(x, "[PFG]: "),# conditions below, for filtering taxa to be labelledrank=="Phylum"|rank=="Genus"&prevalence>0.2# all phyla are labelled, and all genera with a prevalence of over 0.2)key
You can do more with these trees to customise them to your liking. See an extended tutorial here on the microViz website: including how to directly label taxa on the colored plots, change the layout and style of the trees, and even how to use a different regression modelling approach.
---title: "Practical 2A - microbiota analyses"subtitle: "2024 NUTRIM microbiome & metabolome workshop"author: David Barnettdate: last-modifiedformat: htmlkeep-md: falsetheme: light: flatly dark: darklycss: ../../.css/instructions.cssembed-resources: truecode-block-border-left: truecode-block-bg: truetoc: truetoc-location: righttoc-depth: 4toc-expand: 1number-sections: truenumber-depth: 3fig-align: centerfig-dpi: 200fig-width: 7.5fig-height: 5fig-responsive: truecode-tools: truecode-fold: falsecode-link: truelightbox: autolink-external-icon: truecache: true---## Intro**WORK IN PROGRESS** - More restructuring and refinement is planned.## Outline of Part 2A### Research questions#### **Primary aim:**Does the bacterial gut microbiota composition of IBD-diagnosed patients differ from the control patients?- **Diversity:** Is richness or diversity associated with IBD diagnosis?- **Composition:** Does overall bacterial microbiota composition associated with IBD diagnosis?- **Taxa:** Is the relative abundance of specific bacterial taxa (e.g. genera) associated with IBD diagnosis?::: {.callout-note collapse="true"}### Secondary aims (an extra challenge for part 2B)**Activity:**\Is current disease activity level associated with microbiota diversity, composition, or the relative abundance of specific taxa?**Medication:**\Are IBD-related medications associated with microbiota diversity, composition, or the relative abundance of specific taxa?:::### Methods1. **Diversity** - Compare richness and diversity between groups2. **Dissimilarity** - Create (interactive) ordination plots and bar charts - Compare overall compositions between groups3. **Differential abundance** - Compare relative abundance of individual taxa between groups## Setup### Load R packages 📦```{r}library(here)library(tidyverse)library(broom)library(phyloseq)library(microViz)library(ggstatsplot)library(writexl)```### Read phyloseq dataRead the phyloseq we created in part 1.```{r}ps <-read_rds(file =here("data/papa2012/processed/papa12_phyloseq.rds"))```## DiversityOur research questions for this section are:- How rich and diverse is the gut bacterial microbiota of each patient?- Does this gut microbiota richness or diversity differ by diagnosis group?### Richness- The simplest richness measure is just counting, a.k.a. "Observed Richness".- We already computed the observed richness of genera in part 1B.::: panel-tabset#### Plot richness```{r}#| fig-height: 3#| fig-width: 5ps %>%samdat_tbl() %>%ggplot(aes(x = N_genera, y = diagnosis, color = diagnosis)) +geom_boxplot(outliers =FALSE) +geom_jitter(height =0.15, alpha =0.5) +theme_classic()```#### Linear regression / ANOVAYou can use standard statistical testing on the richness values e.g. linear regression or ANOVA```{r}richness_lm <-lm(data =samdat_tbl(ps), formula = N_genera ~ diagnosis)anova(richness_lm)```You could do also standard ANOVA post-hoc pairwise comparisons.```{r}richness_tukey <-TukeyHSD(aov(richness_lm))richness_tukey```#### ggstatsplotCombine group comparison stats and plots in one go with `ggstatsplot::ggbetweenstats()````{r}#| fig-width: 7#| fig-height: 5ggbetweenstats(data =samdat_tbl(ps), x = diagnosis, y = N_genera, type ="parametric", p.adjust.method ="fdr", var.equal =TRUE, bf.message =FALSE, results.subtitle =TRUE)```:::### Diversity**Remember:** a true measure of ecosystem diversity (e.g. Shannon index) will consider the richness *and evenness* of the ecosystem.> A rich ecosystem dominated by only one or two of its taxa is still a less diverse ecosystem than one with an **even** distribution of the same number of taxa.We already computed Shannon diversity of genera and the effective Shannon in part 1B.Here we will use the effective Shannon diversity $e^H$ because of its more intuitive interpretation.::: panel-tabset#### Plot diversity```{r}#| fig-height: 3#| fig-width: 5ps %>%samdat_tbl() %>%ggplot(aes(x = Effective_Shannon_Genus, y = diagnosis, color = diagnosis)) +geom_boxplot(outliers =FALSE) +geom_jitter(height =0.2) +theme_classic()```#### Linear Regression / ANOVA```{r}eShannon_lm <-lm(data =samdat_tbl(ps), formula = Effective_Shannon_Genus ~ diagnosis)anova(eShannon_lm)``````{r}eShannon_tukey <-TukeyHSD(aov(eShannon_lm))eShannon_tukey```::: {.callout-tip collapse="true"}## Save the statistics?You can get a tidy data frame of results using the `broom::tidy` function on various statistical model objects.```{r}broom::tidy(eShannon_tukey)```This can be useful when you need to save your model output e.g. to Excel, to format for a table in your article.```{r, eval=FALSE}broom::tidy(eShannon_tukey) %>% write_xlsx(here("practical-2/test-table.xlsx"))```:::#### ggstatsplot```{r}#| fig-width: 7#| fig-height: 5ggbetweenstats(data =samdat_tbl(ps), x = diagnosis, y = Effective_Shannon_Genus, type ="parametric", p.adjust.method ="fdr", var.equal =TRUE, bf.message =FALSE, results.subtitle =TRUE)```:::## Dissimilarity & OrdinationOur research questions for this section are:- What gut bacterial microbiota compositions are present or common in this cohort?- Does the average overall composition of the gut microbiota differ by patient diagnosis group?### Aggregate ➔ Calculate ➔ OrdinateIn order to create a PCoA ordination - we need to first make two choices:- At which taxonomic rank will we aggregate the counts? (for 16S data, this is usually Genus)- Which dissimilarity measure to use when calculating the distance matrix?### Common dissimilarity measuresYou heard about several commonly-used dissimilarity measures in the lecture. In the sections below, we will calculate a distance matrix with each one, and use each to plot a PCoA and test for diagnosis group differences with PERMANOVA.::: panel-tabset#### Binary JaccardAn unweighted measure - based only on taxon presence or absence.You must remember to run a "binary" transform on your data before computing "jaccard" distance.```{r}psx_jaccard <- ps %>%tax_agg(rank ="Genus") %>%tax_transform("binary") %>%# converts counts to absence/presence: 0/1dist_calc(dist ="jaccard")```::: panel-tabset##### Distance matrix```{r}psx_jaccard```##### PCoA plot```{r}psx_jaccard %>%ord_calc("PCoA") %>%ord_plot(colour ="diagnosis") +stat_ellipse(aes(colour = diagnosis)) +coord_equal()```##### PERMANOVA```{r, message=FALSE}perm_jaccard <- psx_jaccard %>% dist_permanova(variables = "diagnosis")perm_get(perm_jaccard)```:::#### Bray-CurtisBray-Curtis is an abundance-weighted dissimilarity measure.It is probably the most commonly used dissimilarity measure in microbiome research.```{r}psx_bray <- ps %>%tax_agg(rank ="Genus") %>%tax_transform("identity") %>%# the "identity" transform changes nothingdist_calc(dist ="bray")```::: panel-tabset##### Distance matrix```{r}psx_bray```##### PCoA plot```{r}psx_bray %>%ord_calc("PCoA") %>%ord_plot(colour ="diagnosis") +stat_ellipse(aes(colour = diagnosis)) +coord_equal()```##### PERMANOVA```{r, message=FALSE}perm_bray <- psx_bray %>% dist_permanova(variables = "diagnosis")perm_get(perm_bray)```:::#### UniFracThe phylogenetic distance family - has unweighted, weighted, and generalised versions.You must use ASV-level data (i.e. no taxonomic aggregation) and have a phylogenetic tree available.We will not practice with UniFrac distances today, because they can be quite slow to calculate.{fig-alt="Cartoon illustration of phylogenetic tree from: https://www.azolifesciences.com/article/What-is-Molecular-Phylogenetics.aspx"}#### AitchisonThe Aitchison distance is a CoDA distance method - named after John Aitchison, a pioneer in the field of Compositional Data analysis. related to CLR + PCA.```{r}psx_aitchison <- ps %>%tax_agg(rank ="Genus") %>%tax_transform("identity") %>%# the "identity" transform changes nothingdist_calc(dist ="aitchison")```::: panel-tabset##### Distance matrix```{r}psx_aitchison```##### PCoA plot```{r}psx_aitchison %>%ord_calc("PCoA") %>%ord_plot(colour ="diagnosis") +stat_ellipse(aes(colour = diagnosis)) +coord_equal()```##### PERMANOVA```{r, message=FALSE}perm_aitchison <- psx_aitchison %>% dist_permanova(variables = "diagnosis")perm_get(perm_aitchison)```::::::You can see from the PERMANOVA model outputs that the p value is below 0.05. So there is good statistical evidence that the bacterial gut microbiota composition of C-section delivered infants has a different composition than vaginally delivered infants at 4 days of age.::: {.callout-note collapse="true"}#### Dissimilarity or distance?These terms are often used interchangeably.Strictly, all distances are dissimilarities, but not all dissimilarities are distances.A true "distance metric" $d$, must satisfy 3 properties:1. **Identity of indiscernibles**: For any samples $a$ and $b$, $d(a, b) = 0$ if and only if $a = b$2. **Symmetry**: For any samples $a$ and $b$, $d(a, b) = d(b, a)$3. **Triangle** **inequality**: For any samples $a$, $b$, and $c$, $d(a, c) ≤ d(a, b) + d(b, c)$ - **3** means: "the direct path between two points must be at least as short as any detour" - **3** is not true for e.g. Bray-Curtis... but in practice this is very rarely problematic:::::: {.callout-note collapse="true"}#### psExtra objectsmicroViz often creates objects of class `psExtra` which store info about the aggregation and transformations you perform.- `psExtra` can also store a distance matrix (and an ordination or PERMANOVA results)- You can extract the distance matrix with `dist_get()````{r}distances <- psx_jaccard %>%dist_get()as.matrix(distances)[1:4, 1:4]```Notice how the Binary Jaccard dissimilarities range between 0 (identical) and 1 (no shared genera).```{r}range(as.matrix(distances))```:::::: {.callout-note collapse="true"}#### PCoA recap**Principal Co-ordinates Analysis is one kind of ordination:**- PCoA takes a sample-sample distance matrix and finds new dimensions (a coordinate system)- The new dimensions are created with the aim to preserve the original distances between samples- It also aims to capture the majority of this distance information in the first dimensions- This makes it easier to visualize the patterns in your data, in 2D scatterplots 👀**For more info, see "GUSTAME"**There is helpful info about ordination methods, including PCoA, on the GUide to STatistical Analysis in Microbial Ecology website (GUSTA ME). <https://sites.google.com/site/mb3gustame/dissimilarity-based-methods/principal-coordinates-analysis>This website covers a lot of other topics too, which may be interesting for you to read at a later date if you'll work on microbiome analysis.:::## Interactive ordination!`microViz` provides a `Shiny` app `ord_explore()` to interactively create and explore PCoA plots and other ordinations. Let's give it a try!To start the Shiny app, copy these lines to your Console (**not** the Quarto doc!)``` r# Note: we filter out OTUs that only appear in 1 sample, to speed up the appps %>%tax_filter(min_prevalence =2, verbose =FALSE) %>%ord_explore()```::: {.callout-important collapse="false"}##### Unblock popups?!To allow the interactive function to open a new tab in your browser, you may need to unblock pop-ups for posit.cloudIf you don't see anything after running the `ord_explore` command, check for messages/notifications from your browser.:::::: {.callout-tip collapse="true"}##### A few things to try out1. Colour the samples using the variables in the sample data2. Select a few samples to view their composition on bar charts!3. Change some ordination options: - Different rank of taxonomic aggregation - Different distances we've discussed4. Copy the automatically generated code - Exit the app (press escape or click red 🛑 button in R console!) - Paste and run the code to recreate the ordination plot - Customise the plot: change colour scheme, title, etc.5. Launch the app again with a different subset of the data: - Practice using `ps_filter()` - e.g. use the data of only the UC patients' gut microbiota! - Colour points by the dominant genus?``` r# Copy these lines to your Consoleps %>%tax_filter(min_prevalence =2, verbose =FALSE) %>%ps_filter(diagnosis =="UC") %>%# calculate dominant Genus for each sample (optional)ps_calc_dominant(rank ="Genus", none ="Mixed", other ="Other") %>%ord_explore()```:::::: {.callout-warning collapse="true"}###### **Beware: some important notes on interactive analysis****There are many distances available**Feel free to try out distances we haven't talked about, **BUT**:1. You should not use a distance that you don't understand in your work, even if the plot looks nice! 😉2. A few of the distances might not work correctly... - They are mostly implemented in the package `vegan` and I haven't tested them all - Errors may appear in the RStudio Console - You can report to me any distances that don't work (if you're feeling helpful! 😇)**There are several other ordination methods** **available**Try out PCA, principal **components** analysis, which does NOT use distancesWe will not discuss "constrained" or "conditioned" ordinations today:> If you are interested in e.g. RDA, you can look this up in the [Guide to Statistical Analysis in Microbial Ecology](https://sites.google.com/site/mb3gustame/constrained-analyses/redundancy-analysis){target="_blank"}:::## PERMANOVA"Permutational multivariate analysis of variance" - what does that mean?- **Permutational** - statistical significance estimates obtained by shuffling the data many times- **Multivariate** - more than one dependent/outcome variable (i.e. the pairwise distances)- **Analysis of variance** - ANOVA (statistical modelling approach)::: {.callout-note collapse="true"}#### Covariate-adjusted PERMANOVAYou can adjust for covariates in PERMANOVA, and often should, depending on your study design.Let's fit a more complex model, adjusting for sex and age.```{r}ps %>%tax_agg(rank ="Genus") %>%dist_calc(dist ="bray") %>%dist_permanova(variables =c("diagnosis", "gender", "age_years"),n_perms =99, seed =111 ) %>%perm_get()```Use more permutations for a more precise and reliable p.value in your real work (it is slower).Always set a *seed* number for reproducibility of this random permutation method!:::::: {.callout-note collapse="true"}##### Reporting PCoA and PERMANOVA methods- Your methodological choices matter, you should report what you did: - any relevant rare taxon filtering - the taxonomic rank of aggregation - the dissimilarity measure used to compute the pairwise distances - any covariates included in the statistical modelIt's a good idea to decide on one or two distance measures *a priori*, and report both (at least in supplementary material). The choice of distance measure can affect results and conclusions!:::::: {.callout-note collapse="false"}##### More details on PERMANOVASee this excellent book chapter by Marti Anderson: [https://onlinelibrary.wiley.com/doi/full/10.1002/9781118445112.stat07841](https://onlinelibrary.wiley.com/doi/full/10.1002/9781118445112.stat07841){target="_blank"}Sometimes PERMANOVA is called NP-MANOVA (non-parametric MANOVA) e.g. on the GUide to STatistical Analysis in Microbial Ecology [website](https://sites.google.com/site/mb3gustame/hypothesis-tests/manova/npmanova){target="_blank"}:::## PCA"Principal **Components** Analysis"For practical purposes, PCA is quite similar to Principal Co-ordinates Analysis.In fact, PCA produces equivalent results to PCoA with Euclidean distances.### PCA on CLR-transformed taxa```{r}ps %>%tax_filter(min_prevalence =2, verbose =FALSE) %>%tax_transform(rank ="Genus", trans ="clr", zero_replace ="halfmin") %>%ord_calc(method ="PCA") %>%ord_plot(alpha =0.6, size =2, color ="diagnosis") +theme_classic(12) +coord_fixed(0.7)```- After the CLR transformation, the plot looks better- We can see a pattern where the gut microbiomes of infants cluster by birth mode#### So why is PCA interesting for us?- Principal components are built directly from a (linear) combination of the original features.- That means we know how much each taxon contributes to each PC dimension- We can plot this information (loadings) as arrows, alongside the sample points```{r}pca <- ps %>%tax_filter(min_prevalence =2, verbose =FALSE) %>%tax_transform(rank ="Genus", trans ="clr", zero_replace ="halfmin") %>%ord_calc(method ="PCA") %>%ord_plot(alpha =0.6, size =2, color ="diagnosis", plot_taxa =1:6, tax_vec_length =0.5,tax_lab_style =tax_lab_style(type ="text", max_angle =90, aspect_ratio =1,size =3, fontface ="bold" ), ) +theme_classic(12) +coord_fixed(ratio =1, xlim =c(-3, 3), ylim =c(-3, 3), clip ="off")pca```Interestingly, samples on the right of the plot (which tend to be UC patients) seem to have relatively more *Escherichia/Shigella*, and less *Blautia*, *Faecalibacterium* and *Roseburia*.::: {.callout-important collapse="true"}###### Wait, how to interpret these taxa loadings?**In general:**The relative length and direction of an arrow indicates how much that taxon contributes to the variation on each visible PC axis, e.g. Variation in *Faecalibacterium* abundance contributes quite a lot to variation along the PC1 axis.The direction allows you to infer that samples positioned towards the left of the plot will tend to have higher relative abundance of *Faecalibacterium* than samples on the right of the plot.*Bacteroides* variation contributes to both PC1 and PC2, as indicate by its high (negative) values on both axes.**Cautiously**- There are caveats and nuance to the interpretation of these plots, which are called PCA bi-plots- You can read more here: [https://sites.google.com/site/mb3gustame/indirect-gradient-analysis/principal-components-analysis](https://sites.google.com/site/mb3gustame/indirect-gradient-analysis/principal-components-analysis){target="_blank"}:::::: {.callout-tip collapse="true"}#### "Help, what are Euclidean distances?"- Euclidean distances are essentially a generalization of Pythagoras' theorem to more dimensions.- In our data every taxon is a feature, a dimension, on which we calculate Euclidean distances.**Pythagoras' theorem:**$$c = \sqrt{a^2 + b^2}$$**Euclidean distance:**$$d\left(p, q\right) = \sqrt{\sum _{i=1}^{n_{taxa}} \left( p_{i}-q_{i}\right)^2 }$$- Distance $d$ between samples $p$ and $q$, with $n$ taxa.#### Euclidean PCoA```{r}#| code-fold: trueps %>%tax_agg(rank ="Genus") %>%dist_calc(dist ="euclidean") %>%ord_calc(method ="PCoA") %>%ord_plot(alpha =0.6, size =2) +geom_rug(alpha =0.1)```#### PCA on counts```{r}#| code-fold: trueps %>%tax_agg(rank ="Genus") %>%ord_calc(method ="PCA") %>%ord_plot(alpha =0.6, size =2) +geom_rug(alpha =0.1)```##### **Problems with PCA (or PCoA with Euclidean Distances) on microbiota data**- These plots look weird! most samples bunch in the middle, with spindly projections..- Sensitive to sparsity (double-zero problem) --\> filter rare taxa- Excessive emphasis on high-abundance taxa --\> log transform features first##### Log transformations, and CLR- First let's look at the abundance again, this time with heatmaps.- Each column is a sample (from a child), and each row is a taxon.```{r, fig.height=4, fig.width=6}ps %>% tax_sort(by = sum, at = "Genus", trans = "compositional", tree_warn = FALSE) %>% tax_transform(trans = "compositional", rank = "Genus") %>% comp_heatmap(samples = 1:20, taxa = 1:20, name = "Proportions", tax_seriation = "Identity")```- Even though we have picked the top 20 most abundant genera, there are still a lot of zeros- **Problem:** We need to deal with the zeros, because `log(0)` is undefined.- **Solution:** add a small amount to every value (or just every zero), before applying the log transformation.- This small value is often called a pseudo-count.::: {.callout-tip collapse="true"}###### What value should we use for the pseudo-count?- One easy option is to just add a count of 1- Another popular option is to add half of the smallest observed real value (from across the whole dataset)- In general, for zero replacement, keep it simple and **record your approach**:::#### Centered Log Ratio transformation:**Remember**, [Microbiome Datasets Are Compositional: And This Is Not Optional.](https://doi.org/10.3389/fmicb.2017.02224)::: {.callout-note collapse="true"}###### More details on the "CoDa" problem:- The sequencing data gives us relative abundances, not absolute abundances.- The total number of reads sequenced per sample is an arbitrary total.**This leads to two main types of problem:**- Interpretation caveats: see differential abundance section later- Statistical issues: taxon abundances are not independent, and may appear negatively correlated- These issues are worse with simpler ecosystemsExample: If one taxon blooms, the relative abundance of all other taxa will appear to decrease, even if they did not.*There is the same problem in theory with RNAseq data, but I suspect it is less bothersome because there are many more competing "species" of RNA transcript than there are bacterial species in even a very complex microbiome.* *The centered-log-ratio transformation (along with some other similar ratio transformations) are claimed to help with the statistical issues by transforming the abundances from the simplex to the real space.*:::**TL;DR - the CLR transformation is useful for compositional microbiome data.**- Practically, the CLR transformation involves finding the geometric mean of each sample- Then dividing abundance of each taxon in that sample by this geometric mean- Finally, you take the natural log of these ratios```{r, fig.height=3, fig.width=6}ps %>% tax_sort(by = sum, at = "Genus", trans = "compositional", tree_warn = FALSE) %>% tax_agg(rank = "Genus") %>% tax_transform(trans = "clr", zero_replace = "halfmin", chain = TRUE) %>% comp_heatmap( samples = 1:20, taxa = 1:20, grid_lwd = 2, name = "CLR", colors = heat_palette(sym = TRUE), tax_seriation = "Identity" )```::::::::: {.callout-note collapse="true"}###### Fancy circular bar charts?We can make another kind of bar plot, using the PCA information to order our samples in a circular layout.```{r}iris <- ps %>%tax_filter(min_prevalence =2, verbose =FALSE) %>%tax_transform(rank ="Genus", trans ="clr", zero_replace ="halfmin") %>%ord_calc(method ="PCA") %>%ord_plot_iris(tax_level ="Genus", n_taxa =12, other ="Other",anno_colour ="diagnosis",anno_colour_style =list(alpha =0.6, size =0.6, show.legend =FALSE) )``````{r, fig.height=5, fig.width=10}patchwork::wrap_plots(pca, iris, nrow = 1, guides = "collect")```:::## More bar charts- Stacked compositional bar charts using the `microViz` package, `comp_barplot()` function.- Let's start with a smaller subset of the data, just the control group.::: callout-tip### Tip: filtering phyloseq samplesWe can filter the samples like this, using the sample_data information```{r}ps %>%ps_filter(case_control =="Control") # similar to a dplyr filter!```If you forget a variable's levels, check with `table()` or `unique()````{r}ps@sam_data$case_control %>%table()```:::```{r, fig.width=5, fig.height=7, fig.align='center', out.width="70%"}ps %>% ps_filter(case_control == "Control") %>% comp_barplot("Family", n_taxa = 10, merge_other = FALSE) + coord_flip() + ggtitle("Control participants")```### Organising your bar charts- Let's look at all of the data from all diagnosis groups.- We can add the ggplot2 `facet_wrap()` to our plot, to separate the groups.```{r}#| fig-width: 7#| fig-height: 5ps %>%comp_barplot("Family", n_taxa =10, merge_other =FALSE) +facet_wrap(facets =vars(diagnosis), scales ="free") +coord_flip() ```::: {.callout-note collapse="true"}### More bar chart resources:More examples/tutorial of visualizing microbiota compositions using stacked bar charts can be found here: <https://david-barnett.github.io/microViz/articles/web-only/compositions.html>:::## Taxon statsFrom the PCA loadings and barplots below, we have some strong suspicions about which taxa have a higher relative abundance in vaginally delivered infants than in c-section delivered infants, and vice versa, but we can also statistically test this. This is often called "differential abundance" (DA) testing, in the style of "differential expression" (DE) testing from the transcriptomics field.```{r}ps %>%comp_barplot(tax_level ="Genus", n_taxa =12, facet_by ="diagnosis", label =NULL, ) +coord_flip() +theme(axis.ticks.y =element_blank())```### Model one taxon- We will start by creating a linear regression model for one genus, Bacteroides.- We will transform the count data by first making it proportions, and then taking a base 2 logarithm, log2, after adding a pseudocount.```{r}bacteroidesRegression1 <- ps %>%tax_transform("compositional", rank ="Genus") %>%tax_model(type ="lm", rank ="Genus",trans ="log2", trans_args =list(zero_replace ="halfmin"),taxa ="Bacteroides", variables ="case_control",return_psx =FALSE ) %>%pluck(1)```- Looking at the regression results```{r}summary(bacteroidesRegression1)``````{r}broom::tidy(bacteroidesRegression1, conf.int =TRUE)```### Covariate-adjusted modelWe can fit a model with covariates, as we did for PERMANOVA- We will convert the categorical variables into indicator (dummy) variables- We will scale the continuous covariates to 0 mean and SD 1 (z-scores)- You'll see this will make our subsequent plots easier to interpret later```{r}ps <- ps %>%ps_mutate(IBD =if_else(case_control =="Case", true =1, false =0),Female =if_else(gender =="female", true =1, false =0),Age_Z =scale(age_years, center =TRUE, scale =TRUE) )``````{r}bacteroidesRegression2 <- ps %>%tax_transform("compositional", rank ="Genus") %>%tax_model(type ="lm", rank ="Genus", taxa ="Bacteroides",trans ="log2", trans_args =list(zero_replace ="halfmin"),variables =c("IBD", "Female", "Age_Z"),return_psx =FALSE ) %>%pluck(1)```- Looking at the regression results```{r}summary(bacteroidesRegression2)broom::tidy(bacteroidesRegression2, conf.int =TRUE)```### There are many DA methods!- This method simple method is borrowed from MaAsLin2- Note: they call the compositional transformation "Total Sum Scaling (TSS)")- This is quite a straightforward method, so we will stick with this for today- But, many statistical methods have been developed for differential abundance analysesMicrobiome abundance data are quite awkward, statistically speaking, due to their sparseness and compositionality. Each successive method claims to handle some aspect of this awkwardness "better" than previous methods.The aim is to have a method with adequate power to detect true associations, whilst controlling the type 1 error rate, the "false positive" reporting of associations that are not "truly" present.Results are surprisingly inconsistent across the different methods, as demonstrated this year in a [fascinating analysis by Jacob Nearing and colleagues](https://www.nature.com/articles/s41467-022-28034-z).#### So, what to do?- Filter out the noise & interpret results with caution! use multiple testing corrections- Remember it's all relative (abundance)- Try 2 or 3 methods and/or use same method as a previous study if replicating - Avoid Lefse and edgeR? - Beware: Not all methods allow covariate adjustment & few allow random effects (for time-series)### Now model all the taxa!?1. We're not normally interested in just one taxon!2. It's also hard to decide which taxonomic rank we are most interested in modelling! - Lower ranks like species or ASVs give better resolution but also more sparsity and classification uncertainty... - Higher ranks e.g. classes, could also be more powerful if you think most taxa within that class will follow a similar pattern.- So now we will fit a similar model for almost every taxon\* at every rank from phylum to genus- \*We'll filter out species with a prevalence of less than 10%```{r}#| warning: false# The code for `taxatree_models` is quite similar to tax_model.# However, you might need to run `tax_prepend_ranks` to ensure that each taxon at each rank is always unique.psModels <- ps %>%tax_prepend_ranks() %>%tax_transform("compositional", rank ="Genus", keep_counts =TRUE) %>%tax_filter(min_prevalence =0.1, undetected =0, use_counts =TRUE) %>%taxatree_models(type = lm,trans ="log2", trans_args =list(zero_replace ="halfmin"),ranks =c("Phylum", "Class", "Order", "Family", "Genus"),variables =c("IBD", "Female", "Age_Z") )``````{r}psModels```*Why filter the taxa? It's less likely that we are interested in rare taxa, and models of rare taxon abundances are more likely to be unreliable. Reducing the the number of taxa modelled also makes the process faster and makes visualizing the results easier!*::: {.callout-note collapse="true"}###### Notes on filtering rare taxaWe probably want to filter out **rare** taxa, before performing some kinds of analysis.##### Why remove rare taxa?**Rare taxa might sometimes be:**1. Sequencing errors2. Statistically problematic3. Biologically irrelevant##### How to remove rare taxa?**What is rare?** Two main concepts.- Low **prevalence** - taxon only detected in a small number of samples in your dataset.- Low **abundance** - relatively few reads assigned to that taxon (on average or in total)Considering the impact of issues 1, 2, and 3, let's say we are not interested in Species that occur in fewer than 2% of samples, and they have to have at least 10,000 reads in total across all samples.```{r}ntaxa(ps) # before filtering``````{r}ps %>%tax_filter(min_prevalence =2, min_total_abundance =100) %>%ntaxa() # after filtering```- Wow so that would remove **most** of our unique taxa!- What is going on? Let's make some plots!```{r}# make table of summary statistics for the unique taxa in shao19psTaxaStats <-tibble(taxon =taxa_names(ps),prevalence = microbiome::prevalence(ps),total_abundance =taxa_sums(ps))``````{r}#| code-fold: true#| code-summary: Some ggplot2 codep <- psTaxaStats %>%ggplot(aes(total_abundance, prevalence)) +geom_point(alpha =0.5) +geom_rug(alpha =0.1) +scale_x_continuous(labels = scales::label_number(), name ="Total Abundance") +scale_y_continuous(labels = scales::label_percent(), breaks = scales::breaks_pretty(n =9),name ="Prevalence (%)",sec.axis =sec_axis(trans =~ . *nsamples(ps), breaks = scales::breaks_pretty(n =9),name ="Prevalence (N samples)" ) ) +theme_bw()p```So most taxa have a low prevalence, and handful have way more reads than most.Let's label those points to check which taxa are the big time players.```{r}p + ggrepel::geom_text_repel(data =function(df) filter(df, total_abundance >1500| prevalence >0.6),mapping =aes(label = taxon), size =2.5, min.segment.length =0, force =20, nudge_y =0.05)```Those taxa make sense for this dataset of gut microbiota samples.Now let's zoom in on the less abundant taxa by log-transforming the axes. We'll also add lines indicating the thresholds of 2% prevalence and 1000 reads abundance.```{r}#| code-fold: true#| code-summary: Some more ggplot2 codepsTaxaStats %>%ggplot(aes(x = total_abundance, y = prevalence)) +geom_vline(xintercept =10, color ="red", linetype ="dotted") +geom_hline(yintercept =2/100, color ="red", linetype ="dotted") +geom_point(alpha =0.5) +geom_rug(alpha =0.1) +scale_x_log10(labels = scales::label_number(), name ="Total Abundance") +scale_y_log10(labels = scales::label_percent(), breaks = scales::breaks_log(n =9),name ="Prevalence (%)",sec.axis =sec_axis(trans =~ . *nsamples(ps), breaks = scales::breaks_log(n =9),name ="Prevalence (N samples)" ) ) +theme_bw()```- We can break this down by phylum if we add the taxonomic table information.```{r, fig.height = 5, fig.width=8}#| code-fold: true#| code-summary: A lot more ggplot2 code!# don't worry about this code if it's confusing, just focus on the plot outputps %>% tax_table() %>% as.data.frame() %>% as_tibble(rownames = "taxon") %>% left_join(psTaxaStats, by = "taxon") %>% add_count(Phylum, name = "phylum_count", sort = TRUE) %>% mutate(Phylum = factor(Phylum, levels = unique(Phylum))) %>% # to fix facet order mutate(Phylum = forcats::fct_lump_n(Phylum, n = 5)) %>% mutate(Phylum = forcats::fct_explicit_na(Phylum, na_level = "Other")) %>% ggplot(aes(total_abundance, prevalence)) + geom_vline(xintercept = 10, color = "red", linetype = "dotted") + geom_hline(yintercept = 2 / 100, color = "red", linetype = "dotted") + geom_point(alpha = 0.5, size = 1) + geom_rug(alpha = 0.2) + scale_x_log10( labels = scales::label_log(), breaks = scales::breaks_log(n = 5), name = "Total Abundance" ) + scale_y_log10( labels = scales::label_percent(), breaks = scales::breaks_log(n = 9), name = "Prevalence (%)", sec.axis = sec_axis( trans = ~ . * nsamples(shao19), breaks = scales::breaks_log(n = 9), name = "Prevalence (N samples)" ) ) + facet_wrap("Phylum") + theme_bw(10)```**How you pick a threshold, depends on what analysis method you are filtering for!**- alpha diversity: do not filter- beta diversity: relevance of threshold depends on your distance measure- differential abundance testing: stringent filtering, prevalence \>5%, \>10%?:::#### Getting stats from the modelsNext we will get a data.frame containing the regression coefficient estimates, test statistics and corresponding p values from all these regression models.```{r}psStats <-taxatree_models2stats(psModels)psStats``````{r}psStats %>%taxatree_stats_get()```#### Adjusting p values- We have performed a lot of statistical tests here!- It is likely that we could find some significant p-values by chance alone.- We should correct for multiple testing / control the false discovery rate or family-wise error rate.*Instead of applying these adjustment methods across all taxa models at all ranks, the default behaviour is to control the family-wise error rate per taxonomic rank.*```{r}psStats <- psStats %>%taxatree_stats_p_adjust(method ="BH", grouping ="rank")# notice the new variablepsStats %>%taxatree_stats_get()```### Plot all the taxatree_stats!- `taxatree_plots()` allows you to plot statistics from all of the taxa models onto a tree layout (e.g. point estimates and significance).- The taxon model results are organised by rank, radiating out from the central root node- e.g. from Phyla around the center to Genus in the outermost ring.`taxatree_plots()` itself returns a list of plots, which you can arrange into one figure with the [`patchwork`](https://patchwork.data-imaginist.com/) package for example (and/or [`cowplot`](https://wilkelab.org/cowplot/articles/plot_grid.html)).```{r, fig.width=6, fig.height=5}psStats %>% taxatree_plots(node_size_range = c(1, 3), sig_stat = "p.adj.BH.rank") %>% patchwork::wrap_plots(ncol = 2, guides = "collect")```#### Taxatree KeyBut how do we know which taxa are which nodes? We can create a labelled grey tree with `taxatree_plotkey()`. This labels only some of the taxa based on certain conditions that we specify.```{r fig.height=4, fig.width=4.5, warning=FALSE}set.seed(123) # label positionkey <- psStats %>% taxatree_plotkey( taxon_renamer = function(x) stringr::str_remove(x, "[PFG]: "), # conditions below, for filtering taxa to be labelled rank == "Phylum" | rank == "Genus" & prevalence > 0.2 # all phyla are labelled, and all genera with a prevalence of over 0.2 )key```You can do more with these trees to customise them to your liking. See an extended tutorial [here on the microViz website](https://david-barnett.github.io/microViz/articles/web-only/modelling-taxa.html#plot-all-the-taxatree_stats): including how to directly label taxa on the colored plots, change the layout and style of the trees, and even how to use a different regression modelling approach.```{r}# try it out!```## Next! ⏩## Session info<details>```{r}sessioninfo::session_info()```</details>
